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ABSTRACT 

The evaluation of professional support personnel 
(PSP) — counselors, school psychologists, nurses, curriculum 
specialists, library/media specialist*, deans, and athletic 
directors — has received scant attention among researchers, theorists, 
and practicing administrators. This situation may reflect the 
inadequate tools for counselor and PSP evaluation along with the 
professional and legal limitations of direct observation of these 
personnel. However , the PSP Evaluation Model offers a thorough and 
comprehensive alternative to direct observation evaluations by 
drawing on multi-faceted data collection consisting of job artifacts 
collected in a portfolio along with input from multiple sources such 
as students, parents, peers, as well as the counselor or PSP staff 
members 1 self-evaluation. These multiple sources enable the counselor 
to document job performance — one of the seven steps in the PSP 
Evaluation Model. The first four steps involve identification of 
system and program needs, listing of job duties, selecting 
performance indicators, and setting performance standards. The 
remaining three steps, completed in cyclical fashion each appraisal 
period, include documenting performance, evaluation (a process in 
which the evaluation conference is central to the success of the 
enterprise), and improving or maintaining professional service* The 
PSP Evaluation Model should assist school personnel in sustaining or 
improving their services. Contains 17 references. (RJM) 
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Multi-faceted Data Collection; 
The Key to Evaluating School Counselors 
and Other Support Professionals 

Background 

A review of evaluation literature discloses very little concerning the 
evaluation of professional support personnel - all of the certificated non- 
teaching, non-administrative staff (such as counselors) who serve students, 
teachers, and/or other clients. If these educators have been evaluated at ail, 
the process has been rare t Inadequate or both; the appraisal instrument seldom 
reflects the duties of the position. In Wisconsin, a recent survey of 25% of all 
secondary school counselors, for example* revealed that only 17% were 
evaluated on the basis of explicit, written criteria designed specifically for 
counselor evaluation (Gorton & Olemacher, 1987). The largest proportion of 
the counselors in the study (38%) were evaluated with a teacher evaluation 
instrument that included criteria either inappropriate or inapplicable to the 
evaluation of counselors. In this same study, 34% of the counselors were 
evaluated Informally, using no specific written criteria or procedures, and 26% 
reported no systematic counselor evaluation at all. 

Evaluation of professional support personnel (not only counselors, but 
school psychologists, nurses, curriculum specialists, library/media specialists, 
deans and athletic directors) has not received much attention either among 
researchers and theorists or among practicing administrators; this may simulta- 
neously reflect and contribute to the scanty and Inadequate implementation of 



counselor and PSP evaluation. If the literature is scant, so too is support from 
state education agencies. In the 1993 survey of states which updated a previ- 
ous 1988 study, only 14 of the 41 states responding indicated that they provide 
guidelines to local schools regarding counselor evaluation; and only nine (9) 
states reported that they provide training for evaluators of school counselors 
(Stronge & Tucker, 1994). 

There is, however, a model for evaluating counselors and other support 
personnel that was developed first for the Illinois State Board of Education 
(Stronge, 1988). This model was used by the Illinois Administrators Academy 
to in-service administrators with programs delivered by regional education 
agencies in response to a state mandate. Known as the PSP (Professional 
Support Personnel) Evaluation Model, it has since been elaborated on and 
presented in a larger theoretical and historical context in Stronge and Helm's 
Evaluating Professional Support Personnel in Education (1991 ). These two 
researchers have also been participating in Project CREATE (Center for 
Research on Educational Accountability and Teacher Evaluation), funded by the 
U.S. Department of Education's Office of Educational Research and improve- 
ment. In this project, they are working to refine the PSP model based on 
feedback from both evaluation experts and professional educators In the field, 
which is one reason for presenting this symposium today: in part to disseminate 
information about the model as it currently exists and in part to obtain additional 
feedback concerning the perceived strengths and weaknesses of the model. 



The PSP Evaluation Model contains little that is startiingiy new or 
different, when each of the components is considered individually. We have 
drawn heavily from the best evaluation theory found in the literature and from 
the bast practice uncovered by surveys and literature searches. And while the 
model was developed specifically for professional support personnel, we believe 
it offers a uniquely strong approach to the evaluation of all educators - class- 
room teachers and administrators as well as counselors and other professional 
support personnel. 

The 7 Steps of the PSP Evaluation Model 

The PSP Evaluation Model consists of seven (7) steps. It is, in keeping 
with sound evaluation theory, cyclical in nature. That is, the last step in the first 
evaluation cycle becomes or leads into the first step of the next cycle. We wish 
to emphasize, however, that the first four steps are primarily a one time exer- 
cise. While they require a major investment of time and energy, once complet- 
ed, they need only be "revisited" periodically in order to update in light of 
changes in the school/district's needs, the program, or the position. 

'Step 1 Is to identify system needs. This is essential because the 
philosophy underlying the model is that all personnel should be performing in 
ways that will enable the school (district/program) to achieve its goals and 
mission. Therefore, the system and program needs must be clearly identified. 
More specifically, this entails 1) examination of current programs and person- 
nel and 2) assessment of future needs, both short term and long term. Most if 



not all of these endeavors are completed with some regularity to meet require- 
ments for state approval and/or accrediting agencies. 

'Step 2 Is to Identify the duties of the position. Good performance 
evaluation In the 1990s bears little resemblance to performance evaluation of a 
couple decades ago or more when evaluators focused primarily on the "traits" 
of the teacher or educator, such as appearance, enthusiasm, dedication, etc. 
And although much teacher evaluation in the 1980s was based (somewhat 
erroneously) on Madeline Hunters effective teaching behaviors and elements of 
lesson design, this approach cannot be implemented with counselors and other 
PSP staff because it still relies primarily on direct observation of classroom 
teaching. In fact, one recent national survey (Educational Research Service, 
1968) found that 99.8% of the public schools responding used direct classroom 
observation as the primary data collection technique. 

Sound performance evaluation for counselors and other professional 
support personnel simply must be applied to the duties an individual performs in 
the context of his or her job. The list of duties should be determined jointly by 
the school counselor and the supervisor - either a principal or a personnel 
director/assistant superintendent. This list will in most instances be a more 
detailed compilation of duties derived from a job description, which Is likely to 
be too general for evaluative purposes. 
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Typical duties for school counselors might include, in the area of admin- 
istration, for example: 

♦ Implements and coordinates schoolwide counseling services and 
activities. 

♦ Maintains an organized, functional, current counseling center. 
Or in the area of consultation, some sample duties might include: 

♦ Presents instructional/informational programs to groups of 
students, parents, teachers, and other school and community 
personnel. 

♦ Interprets test data to assist school staff with curriculum planning. 
Once a representative list of significant duties has been constructed! the 
counselor and supervisor are ready for the next step in the evaluation process. 

*Step 3 Is to select performance Indicators. Performance Indicators 
are the "observable, measurable units of performance that can be document- 
ed...." (Strange & Helm, 1991 , p. 130). They are selected on the basis of their 
representativeness of broader categories of performance. Are they like perfor- 
mance objectives or behavioral objectives? If not, how are they different? 
Perhaps when combined with the performance standards set in step 4, they 
share many characteristics with behavioral or performance objectives. Howev- 
er, too often in the past, the latter have been an end in themselves, whereas in 
the PSP model, the performance indicators and performance standards can 
only be a part of the larger process, i.e., the means to an end, which is clearly 
identified from the beginning as Improved performance for the benefit of the 
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larger system and the students it serves. Some sample performance indicators 
(Pis) for the duties listed above include: 

DUTY: Implements and coordinates schoolwide counseling 
services and activities. 

Pt Organizes and conducts special events such as "career day, 99 
"college night" 

DUTY: Maintains an organized, functional...counseling center. 

PI: Displays Information on bulletin boards, tables, ate 

DUTY: Presents instructional/informational programs.,.. 

PI: Conducts programs for Improvement of student study skills; 
OR Conducts teacher In-service based upon assessed need. 

DUTY: Interprets test data to assist school staff with cumculum planning. 

PI: Assists in curriculum revision based on tost results. 
The number of performance indicators utilized with a given duty will vary with 
the importance and nature of that duty. The important consideration is that 
both counselor and supervisor develop these performance indicators jointly and 
that those developed are the most reflective indicators that the duty has been 
performed. 

'Step 4: Set performance standards. A standard is a predetermined 
level of desired or acceptable results (Strange & Helm, 1991, p.146). It must 
be stated in terms of observable , measurable behavior; like a behavioral 
objective, it should whenever possible include specifics such as location, time 
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allotment, accepted rate of accuracy - with the caveat that such specifications 
must be justifiable and realistic, not arbitrary. Too often, this step is slighted in 
performance evaluation. But after identifying the duties and the indicators of 
performing those duties, it is essential to specify not only whether the duties 
have been performed but also how well they have been performed. This 
process requires careful consideration and the greatest clarity possible* The 
choice of terms - e.g M unsatisfactory or needs improvement; satisfactory or 
meets expectations; excellent or outstanding or exceeds standards - may be 
mandated by the state or by local school board policy. Given the terms, 
however, the counselor and supervisor will need to define those terms contextu- 
ally and operationally. What level of performance constitutes "satisfactory" 
performance or Unsatisfactory* performance? The answer to this question lies 
in the identification of those factors mentioned above; it will include both 
quantitative and qualitative considerations. 

"Stop 5 Is to document lob performance. Here is where the PSP 
model offers a viable alternative to the teacher evaluation procedures that rely 
so heavily on direct observation, which is, of course, of limited usefulness in 
counselor evaluation. In documenting job performance, multiple sources of 
data will be collected. These sources fall into two categories: 1 ) job artifacts, 
collected primarily by the counselor in a portfolio and 2) input from other 
individuals with whom the counselor has contact during her or his job perform- 
ance, 



The documentation will be collected throughout the appraisal period by 
both the counselor and the supervisor. If both maintain folders specifically for 
that purpose, they can simply enter documents, notes, or survey instruments as 
those items are developed or used. It is important that they include only those 
artifacts or other items that directly document some aspect of the counselor's 
performance - and generally, items that significantly document that perform- 
ance. The importance of guidelines for including and for evaluating the docu- 
mentation are discussed below. 

"Step 6 is to evaluate performance. Too often in many school evalua- 
tion procedures, this is either the first or second step of the process. In the 
PSP evaluation model, however, the evaluation is the heart of the entire 
• process. The vehicle for conducting the evaluation is the evaluation conference 
- called a performance review or performance appraisal Interview in the 
business world. (Whik* this writer has always had reservations about applying 
business principles to the field of education, it does appear educators have 
much to learn from the business world about conducting evaluation conferenc- 
es.) An evaluation conference may be a part of either formative or summative 
evaluation - preferably both. It should be one component of on-going feedback 
from the sivervisor about the employee's performance rather than an annual, 
one-time event (Johnson, 1979; King, 1984; Buzzotta, 1989; Zemke, 1991; 
Longenecker & Goff, 1990). When this is the case, the evaluation conference 
will hold no surprises and should cause little if any anxiety for either the 
evaiuator or the evaluates 
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The purposes of an evaluation conference are multiple, for the confer- 
ence offers an opportunity to look at both the past and the future of both the 
individual evaluatee and the larger program (i.e., counseling) and school/district. 
The discussion will focus on goals and objectives previously set, and the 
possible reasons some of them might not have been fully met It provides an 
opportunity for the supervisor to recognize good performance and communicate 
suggestions for improvement If there is trust between the individuals, misun- 
derstanding and misinformation can be prevented or corrected during this 
conversation. An effective evaluation conference will also be used for problem 
solving, as the two individuals discuss the job in the context of the organization 
and its mission and needs. Obviously, if the employee is having performance 
difficulties, the conference provides a more formal means of warning the 
individual and offering recommendations for remediation. Finally, the evaluation 
conference and the completed evaluation instrument that will reflect the discus- 
sion provide a permanent record of the employee's performance at that point, 
offering a legally defensible basis for personnel decisions. 

•Step 7 Is to Improve/maintain professional service. The culmination 
of the appraisal cycle is, by definition of a cycle, also the entree into the follow- 
ing evaluation cycle (Stronge, 1994). Following the final, summative evaluation, 
the counselor and supervisor have a clear picture of the counselor's past 
performance and the areas which need additional effort, reflecting either room 
for improvement in the counselor's performance or newly identified system 
needs. 



Encompass d within this step are a variety of personnel decisions 
ranging from termination of unsatisfactory employees, transfers, or provision of 
assistance to marginal employees, to individualized professional development 
plans that expand the counselor's knowledge base or skills level* It may also 
involve setting new goals or objectives or redefining programs or job descrip- 
tions. 

Multi-Faceted Data Collection 

The "job artifacts" (McGreal, Broderick & Jones, 1984) are documents 
and records developed and/or used by the counselor in the course of fulfilling 
his or her duties. Such artifacts might include, for example, logs of student 
appointments, handouts and lesson plans for group counseling activities, forms 
designed by the counselor for use in obtaining or recording information, reports 
written and submitted for district or state use, and calendars or schedules of 
activities* Input from individuals with whom the counselor or other PSP staff 
member works can be obtained through questioning, either with a survey 
instrument or by Interviewing. These constituencies may include peers (teach- 
ers and other professionals with whom the counselor works directly), students, 
and parents, as well as the counselor being evaluated. The assumption, of 
course, Is that only those having direct experience with the counselor in his or 
her job capacity would be surveyed or asked for Input. All respondents should 
be assured of confidentiality; only the compiled results of any surveys would be 
shared with the counselor by the supervisor who collects the information. 



11 

The idea of including input from students, parents and other teachers 
ma,' at first be disconcerting to counselors accustomed to evaluation only by a 
supervisor, if they have been evaluated at all. It may also seem a bit foreign to 
the principal or personnel director responsible for evaluating the counselor. 
However, the use of multiple sources of input and assessment strengthens and 
enriches the final evaluation, as is evident for the following reasons: 

1. Multiple sources increase the validity of the evaluation. 

2. Multiple sources Increase the reliability of the evaluation. 

3. Multiple sources decrease the subjectivity of the evaluation. 

4. Multiple sources expand the performance profile. 

5. Multiple sources facilitate evaluation of quality as well as quantity. 

6. Multiple sources build a more legally defensible record for personnel 

decision-making. 

7. Multiple sources, in short, are more appropriate for professionals 

whose jobs are multi-faceted. 
[For a useful discussion of the advantages and potential liabilities of involving 
each of the five parties in evaluation (supervisor, peers, subordinates, self, and 
outsiders), see Cummlngs & Schwab, in Baird et al., 1982; and Mohrman, 
Resnick-West, and Lawler III, 1989.] 

HpwtgPvtfyaft ih* Pate 

Sound evaluation theory in generai and the PSP Evaluation Model In 
particular require that standards and performance criteria must be established 
prior to any documentation. The counselor or other PSP staff member must 
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know, preferably from the first day of employment, on what basis he or she will 
be evaluated This requirement not only reflects the good faith and fairness of 
the supervising personnel but protects the supervisor, the school administration 
and the school board in the event of a legal challenge to any employment 
decisions based on performance evaluation. 

Job Artifacts The quality of the job artifacts should be determined by 
three considerations (McGreal, Broderick & Jones, 1984): 

♦ content 

♦ design 

♦ presentation 

In looking at content (meaning), the evaluator will consider such factors as the 
validity, appropriateness, relevance of the artifact to the task, ability of the 
artifact to motivate the receiver/user to cooperate or participate, and finally, the 
clarity and conciseness of the artifact. Assessing the design of an artifact 
requires judging the appropriateness of the medium used, the relevance and 
meaningfulness of the artifact for the objective it was designed to achieve, the 
sequencing of the artifact, Its ability to Invite or engage the receiver, and the 
existence of a plan for evaluating the artlfact's effectiveness. Finally, the 
evaluator must consider the presentation - the physical and aesthetic aspects 
of the artifact, as well as directions for Its use. Is the artifact suitable for the 
time allotted? Are directions clearly explained? Is the artifact uncluttered and 
visually attractive? Does It present all the significant details? In all of these 
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three major factors (design, content, presentation), however, the evaluator must 
always look for a connection with the counselor's duties; each artifact must 
document the fulfillment of some job responsibility or duty* And the total 
"portfolio" should reflect some principle of organization, rather than be haphaz- 
ardly thrown together. 

Surveys When the supervisor is assessing the data obtained by question- 
ing individuals (from several constituencies) with whom the counselor works, 
s/he must look at "the forest, not the trees" That is, the supervisor will try to 
identify overall strengths and areas needing Improvement, keeping in mind that 
there are always some individuals providing Input who are consistently critical of 
everyone or who might have a personality conflict with the counselor or who 
might have had some professional disagreement - any one of which could 
affect their objectivity. However, to the extent that the instrument for obtaining 
such feedback focuses on specific counselor duties and performance indicators 
rather than on personality traits, the above named situations will have minimal 
negative impact* Also, those surveyed for Input should Include only those with 
first hand contact with the counselor; whether all Individuals In a given constitu- 
ency or a representative sampling of that constituency Is surveyed will depend 
on the number of persons involved* Finally, we add the reminder that if the 
survey Instrument or list of questions hat been jointly developed by the coun- 
selor and supervisor, both should feel reasonably comfortable with the data 
obtained with the form. 
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The Evaluation Conference 

While the guidelines for conducting an effective evaluation conference 
are really the subject of another presentation/article, we will offer a brief 
overview here to stimulate some awareness of the possibilities. To ensure a 
successful evaluation conference, not only should the evaluator prepare in 
advance but s/he should assist the counselor or PSP staff member in preparing 
also. 

How do they prepare for an evaluation conference? The evaluator 
establishes a mutually convenient time, date and place for the conference, 
preferably at least a week ahead of time. He or she suggests that the counselor 
conduct a self-appraisal, perhaps with a copy of the evaluation instrument. The 
counselor may also be asked to identify or highlight major accomplishments, 
goals achieved, and obstructions to effective performance during the evaluation 
period. The artifacts or documents collected by the counselor during the 
appraisal period should be submitted to the evaluator for review prior to the 
evaluation conference. 

The best evaluation conferences, according to the literature, are "those in 
which the employees do most of the talking and accurately appraise their own 
performance* (Webb, 1989). This, of course, presupposes an understanding of 
the structure of the conference, which focuses on specific achievements, 
behaviors, deficiencies, and the context in which employee goals or tasks were 
met or not met it also presupposes careful planning, collecting and organizing 
of information and preliminary analysis of performance. The supervisor's ability 

* 
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to ask open-ended questions that begin with "When" "Which" "How many" or 
"How" (Alexander Hamilton institute, 1989) will elicit more meaningful responses 
by the evaluatee. "Please explain that," "Give me the details," or "Tell me 
more about that," will draw out a reticent employee. 

The conference should begin with the employee's successes and 
strengths, but when it comes time to shift to problem areas, the evaluator might 
ask "Which parts of the job gave you the most difficulty? And why?" Or, 
continuing to focus on the situation rather than the individual, the supervisor 
might ask, "Which objectives did you have the most difficulty fulfilling? What 
conditions or events made it difficult?" (Stronge & Helm, 1991, p. 208), If the 
counselor is perhaps relying too heavily on excuses based on external factors, 
it may be necessary to shift the line of questioning: "Is there anything you might 
have done differently to bring you closer to achieving this objective?" Ali this 
emphasis on evaiuatee talk may seem foreign - if not outright strange - to 
educators who are used to heavily supervisor dominated conferencing. Yet the 
business and human services sectors almost universally recommend that 
employee talk comprise anywhere from 75%-90% of the conference (Alexander 
Hamilton Institute, 1989; Umiker, 1992). 

Structuring the evaluation conference as described above has the 
advantage, in most instances, of giving the employee an opportunity to cite his 
or her deficiencies and areas needing Improvement. Were the same perform- 
ance problems cited by the evaluator, the employee would be more susceptible 
to the natural tendency toward defensiveness. This approach also raises the 

17 
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comfort level for the evaluated since few actually enjoy directly confronting a 
subordinate about any performance problems. Nevertheless, it should go 
without saying that the supervisor will compare the counselor's self-appraisal 
with that of the supervisor him/herself. Accomplishments and successes should 
receive warm recognition and approval; if improvements are in order, the 
evaluator may ask "What can we do to improve your performance?" or "What 
would you like to work on during the next appraisal period to enhance your 
performance further?" (Stronge & Helm, 1991 , 209). Perhaps goals and 
objectives for the next appraisal period will be jointly determined at this time. 

It hardly needs to be added that the dynamics of the evaluation 
conference will vary, with the ages and amount of experience of the counselor 
/employee and the supervisor, as well as with the level of competence demon- 
strated by the employee. Whatever the specifics, however, when the evaluation 
conference has concluded, both Individuals should have a clear idea of the 
assessment to be made by the evaluator. The evaluation form should only be 
completed in its final form by the evaluator after the conference. 
SUMMARY 

In the evaluation of school counselors and other professional support 
personnel, direct observation has serious professional and legal limitations. 
The PSP Evaluation Model offers a thorough and comprehensive alternative to 
evaluation based on direct observation: multi-faceted data collection, consisting 
of job artifacts collected in a portfolio end input from multiple sources such as 
students, parents, peers (teachers and other educators) as well as the counsel- 
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or or PSP staff member's self-evaluation. The data collected from these multiple 
sources enable the counselor to document job performance, which is one of the 
seven steps In the PSP Evaluation Model. The first four are primarily one time 
activities, involving identification of system and program needs, listing of job 
duties and performance indicators and setting standards for judging the quality 
of the performance. The remaining three steps, completed in cyclical fashion 
each appraisal period, include documenting performance, evaluating (a process 
in which the evaluation conference is central to the success of the enterprise), 
and improving or maintaining professional service. The PSP Evaluation Model, 
in short, is designed to assist school personnel, both counselors/professional 
support personnel and supervisors, in achieving the goal of the organization: to 
continually sustain or improve its educational programs and services. 
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